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ABSTRACT zs 7 ' 
A process. assessuent vas corducted to deteveine the 
‘effects iweness of the proceduré | used by the Bureau‘of Occupational “and 
Adult Education, 0.S. Office of 2ducation (USOE), in conducting State 
Managerent EvaluationsReviews for Compliance (MERC) with Pede'ral 
legislation and ide gr gre: public vocational and adult 
‘educatipo. programs. The purpose wag to assure that .the continaation 
' of the MERC process be of greatest value to the future relationship 
between| Federal any State agencies. The objectives of this assessaent 
were: (1) To determine the feasibility of continuing the program, (2) 
to.detelrmine ways to improve the process, (3} to identify cost to . 
. States ifor the review, (%) to identify benefits to the States, and > 
“{S), to identify benefits to USOB. Otilizing a structured interview 
“+ approa h, Sierra Planning and Research Associates (SPRA) contacted 
_-". personnel in. 5S of the 13 States which had been reviewed by MBRC. SPRA 
‘ . also intergiewed persounel in six regional offices and various 
personnel From the central OSOZ. Additionally, SPRA observed the MERC 
: review /in Idaho. A total of 97 persons veré interviewed by SPRA, and 
ae data from these various soutces yielded 14 récompendations which ar® 
-/ . categotized as follows: (1) The MERC: process should be better f 
organizéd, (2) communications should be inproved asong all personnel, ° 
» €3) the MERC process should be reviewed periodically tq reaove or 
‘s alter unsuitable items, and (4) -the process should be systematized 
throug the developswent and usage af a ocedures mannal. The general 
conclusion of the investigators is that the AERC process shoaid | .be 
continued: on a regular basis as practically all thq@ jpeople 
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TNTRABUCTION- +3 4 ms 
ete oon eit a : 
ssment has been aonducted ‘to determine ‘the 
rocedure used'by the Bureau of tye ei 
SOE, in conducting State“Management. fvai 
liance (MERC) with federal tegislation and 
pubVic vocational and adult education pro- 
sure’ that thé continuation of the -Manage- 
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ment fonship. pytween f be.of greatest value to the’futura, 9 / ' | 


ty relationship b 
of the process ‘at the 
at . ducted. “ 


3 On September 17, 
“+ was submitted to. the,B 


a privates cys ting firm. .~ 3. 1 fe! 


| As*shown inthis 
genérélly supportive o 


in 1] States, Puerto R 


eee are given to further enhance future MERC acti 


Scope of: the Assessment a es . a 


To. dite, -8 review for compliance has beer conducted 


tween federal and state agencies, this, evdluation 


initial stdges of imp lementatton was con 


ote 


nc ee -@ og te 
1976, a proposal to assess the-MERC p 
ureau.of Occupational and ‘Adult Educatjion, | 


, _USOE, by the Sierra Planning and-Research Associates (SPRA) | and ~~ 
-—~.  , Of October 1, 1976, th 


is assessment proposal was awarded to/SPRA, 


- t 7 a om, ‘ . j * 
refort, the-results of this assessmeqt are'- 
f the:MERC :précess. . The findings an@ recom- 
ies. ; . 


igo, and the Virgin Islands. .Fourjof - 


‘,  thése States and Puerto Rico were selected for investigation by. ; 


SPRA. ~ Additionally, personnel in regional-offices and the 
central office. were ‘interviewed so that three populations, or, « 


. 


87 persons, were contacted for this investigation: State, - . 
Regional, and U. S. Office. In addition, the principal /investi- 


-~ “Qator accompanied the MERC. team in the review. at Boise, 


_ on November 4/1976. 


7 Speck fically, SPRA investigators made the followin 
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Offices at Seattle, New York; Bodton, 


Kansas City, Denver, and San Francisco. ; ' 

. eos < . . R * . ~» ; ; ‘i es 7 : 2 ; 

oa . 2. State Offices at~San Juan, Puerto Rito; Augusta, ; 
ae ; Maije; Trenton, New Jersey::Des Moines, Lowa; 
“ w~ang Denver, Colorado. Coma aE 
3.. “MERC review at Boise, Idaho. a 
“ a of. ; . il aes 
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2 *. Ob, ectités ‘of ‘the Assessment . e fe ag 
aoe 


a The jversis objective of this assessment was to ‘evaluate. : 
thé. jal process and to'providaga basis for re mendations ~ * Sd 
ctt’may be implemented ‘through state mandgeftent, evil uat ions a’ 


7 “prepared “by forthcoming legislation. hs o a 


e, 
“ : specifically, the objactives of this assgssnent are: ; : ; 
2 * : . ' @ 


eT. . bo Assess the State Managenent. ‘Evaluation Reviews ‘ 
forTompTance -(MERC). to determine the relative - 


sw. soutidiess - dnd -Rpas ibility of ee une the - Page ge, 
“7 program, = y oS . ee “y 
a rs ‘ 4 7 By ~y 
Binie 8 De termine’ ways in which the NERC process ‘can 
1 het beimprovid.e © , Y 
aie Pa tt, ; ‘+ ‘ Fe : “ sea Zz 
eat ow 3. Identify state costs ins preparing “for, ah: i. 
‘ r oe participating ip, the MERC program, oe 


"sg 4. Identify the bene its to the states asthe 9. cal 
aie result. of the MERC visits already conducted. , .. 


yr 


‘ € a, Identify the penstits of, the MERC Dien. to, Phy 
og we the USOE. +, we EIT 


toe Overview of ‘the Assessment I por gs 


In the following chapters the ; ares of tfids assessment . .-. 
Y °° ate described: an: analysis of the Its of oe findings is ne 


to given; conclusions, which are spe from the ; ta,-are made; -and 
: peccrmendet lone Yee given to saubport. and ephe ire the de process. 
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ne eS CHAPTER -2 


a ae || PROCEDURE OF THE ASSESSMENT ' 
: is . \ : oe on 
the procedure for conducting this assessment’ was ‘personal. 8, ~ 


. lizina an interview/questionnaire form. Those interviewed were — 

‘persosis who had participated in MERC reviews. -Additionstly, . °-. : 
the‘orincipal investigator accompanied the MERC-team on the 
review at Boise, Idaho in order to obtain first-hand knowledge 

of the implementation and operation procedures ‘of the MERC. 

process, , vo 


contact slg ae 224) Office, Regional, and ‘State personnel. uti - 


In addition tc the field work, investigators reviewed the 
documentation of MERC procedures and reports. 


7 ” Preparation of Forms . 


In order to assure consistency in the conduct of the inter- 

views, two questidimaires were. developed by SPRA. Cre question- 

. faire was designed for Central’ Office and Regional Office per- ; 

‘Sonnel who had conducted MERC reviews, and a second question-— : ae. 

naire was developed which would be appropriate for state person-- - 
nel who had been -reviewed by MERC. ; 


a oe OF Items of the questionnaires were designed to elicit re- 
: sponses concerning the purooses of MERC, scheduling, orpcedyres 
training and conduct af MERC teams, consistency betw exit . 
"and final reports, and the overall attitude toward the purpose. . . 
and processes of this type of review.. 0 fons were construc- 
ted to force a "yes" ones ay Saeed to ms and to rate the - vee, oe 
S 


e . 


quality of "yes" responses. a} “good,”. "fair," or “poor.” These 
forms were approved by the USQE. — - j ; ‘ 


i ‘ oe Ne — a ee 
In order to assure the clarity and useability of -the “ques ~ 
tionnaire, Mr.’ Courtney Riley, State Director of VocatgonaT . 
Education, arranged to fieid test the ques*jonnaires os rh, . 
appropriate nevada sYa(e Department personnel. As a Teagtt of et 
this fipld test, some revisions were ‘madé in the wordingsef _ a - 
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'. , Guestions: — .. ae 
nae : ase a * 7 2 & & “a 
., Site Visitations ~~ a oa ef ; 
& : me 


Investigstor HubeK mafie contact with #1 Céntral Office per- 
ram sunnel and interviewed.” ederal. MERC participants from six. 
' Regional Offices inorder to ‘determine “the effectiveness of the 
. Program from the federal pgint of view based on their. experi- ; 
. i ence'as MERC team members.” - . Fe eee 
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Investigators Davis and Trout arranged through state-di- - 
rectors to interview stafe MERC pavticipants in four states and 
* Puerto Rico. These site visitations were conducted in order to | 
verify final reports and findings., and-to determine the effective- 


s 


ness of the program. “A total of 46 state personnel was inter- = | © \.° 
Viewed by Drs. Dayis and Trout. ” ae Se. a 
. pu i . are 4 . s P . +e * Lo 
+ + "Participation; in MERC Visit: In order -to obtain’ first-harid ' 
+ knowledge of the Implementation. and operation of the MERC pro- 7 


cesta ot ine ine! sagves ante Davis met with a representative. 
from.the Central Office and the Assocfate Regional Commissioner 
-.an Seattle orior' to. the Idaho pre-reyibw. The next dgy investi- 
gator Davis, Central Office: and Regional Office ‘personnel met — 
‘withthe State Ovrector and his staff in Boise, Idaho in a pre- 
‘review meeting to drovide the state staff with the procedures 
a “which woul'd be used during: the-review. Three. weeks: iater the- 
. Lt. review was made, and the MERC team carried out ‘the activities °~ 
necessaryto measure for compliance. Principal investigator 
Davis was thus able .to observe first-hand the review procedure 
_* Which provided an excellent view of the-entire process. | 


Review of Documents” 2 ope ae 
in ” The OE “internal documents were found to be adequate; however, 


_ there appears to besa Leck of coordination between the various 
documents *developed by Adult £ducation personrel CETA personnel, 
and other. Central Otfice personnel’. Each program has developed _ 

_-~ I, $tatements of MERC principles: process orefiles, and process — 
. : recommendations whith dq not appear, to. receive proper attention, ~ 
r irae? * . ra ‘ rt 


Some. final report documents were reviewed and the format was ay 

-adequate. to provide states with data required to direct them to- “s.. og. 

; _ward compliatce within a workable time frame. “However, a con- me 4 

- siderable 1aq was observed #n-returntng the MERC reports in. ~ 

final form to the states. FS 
x * . : 


Nrdering of Findings: “ ae 7 

“5 _ Responses of completed interview forms were tabulated and 

>" " total responses asd:percent cf responses to each item were _ * 

ee: calculated; Threé‘populations and several sub-populations were _ 
" tdentified for analysis and reporting of responses as. follows: : 
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‘ Se 7 a: 1. Central Office Personnel o | wee , 
1 = 2, Regional Office Personnel ge nat ee 
. i w (a) Adult Eaucation Personnel .in a 
: \ "+ Central Office and Regional Offices 
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: 3. Total. State Personne} ; set 
a s ‘ ~ (a) Colorado Persopnel - 
: - (b) Towa Pérsonnel 
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or ae CHAPTER 3. n 
: 7 FINDINGS OF THE: ASSESSMENT . 


In this chapter is a report of the findings of this assess- 1 
ment Gf the MERE process. These findings ar® shown by an itent 
arralysis of. the responses of the populations (Ne87), and the 

. comments .of these respondents”. oe 


Description of the Populations 


_ Central Office Personnel: The 21 persons interviewed in . 
this population tnclude one representative of: ~ National Ad-, 
visory Board, three CETA personnel, four representatives fron” 
adult education, and two representing Title IX.” 


a 


‘ Regional Office Personnel: The 20 interviews conducted for 
this population include four persons in Region 7, three in Re- 
gion [I], three in Region VII, five in Regton ViIt, three in Re- 
gion IX, and two in Region X. ; “- 


, _ Adult. Education Personnel: Of the areas represented by 
versonnel. ingerviewed tn the Central Office ang the s¥x Region-- 
al Offices, only those in Adult ‘EGucatios were of sufficient. 
number (N=8) to warrant separate analysis- : 


) .. “personnel in State Offices: The total State population of 
- -46 is made up of efght persons in Cotqrado, Six. in lowa, 12 in 
Maing, eight in New Jersey,”and 12 in Puerto Rico: . 


aad Item Analysis of Interview Items | 


. . The number and percent of responses ta each item were cal- 
culated in order ‘to analyze the findings Of ‘this asseSsment. 
- Yes responses were rated “good,” “fair,” Or. "Poor" and ‘the per-, 
cent of these ratings was.calculated from the number "esponding 
"yes. No response and not applicable responses were combined énd 
the percent of these responses was also calculated. AV pee. 
_ centages.were rounded to the nearest whole nunber. a 
© oe Meese. : . = 
hs - Responses of Central Office Personnel: Al] responses of | 
- . this sopnTation (N21) are given in Table 1. Appertdix C.- Follow- 
ing are responses which are of papajcular signj ficance, 


Item 1:% “Was the MERC review teat trained in a manner” 
-w to adequately determine COM} iar ce?" 
» . , — 
a 96 pércent answered yes to this question, and of 
_.* these 19 respondents less tha half (47 percent) 
rated ‘the training aS qagod... «. ae 


( 


F ; { : \ : 

; a | ltem 2: “Was the agenda’ used for MERC aopropriate 

' . ~ for the, tasks?" ae 
90 pencent answered yes to this question’, and of 

these 19 respondents 58- percent. rated: the agenda good. 


Item Je “Were other MERC tasn members courteous and 
» professional in their approachato the tasks?" 


100 percent answered’ yes, to this question, and 
of these 21 respondents 76 percent rated the team' s\ 
approach as good. ; 


- 


Item 4: “Was the ee of the MERC team saber: 
\_ adequate for the job to be accomplished?” — 


: - , °* ‘+ 100 percent answered yes to this question, and of 
ws ‘ these 2l respondents 52 percent rated adequacy of 
the team as qood. - ee 
Item 5: "Was there an adequate iunber of persons in 
the review teams who were: representative of . 
regional ; state, and national personne!?" 


100 percent answered yes to this question, and of. 


*, | these. 21 respondents.81 pertent rated the ‘number -and = 


-reqresentation of the team as 909d. 


sh Item 6: “Did MERC team member's’ you dealt with know 
rs - what they were to do: to carry out their role 
* efficientiy2" - 


: . 86 percent answered yes to, this. question, and Pet 


these 18 respondents 61 percent rated the ef Rictency 
_ of the team as good. gos 


Item 7 cr "Was the. review team forthright in disclo- 
; sures. of problems where Gop tanice, was iy 
“ doubtful ? ie ; ; 
: 7 90 percent answered yes to. this question, and.of 
_—_ _these+19 respondents 84 percent rated the forth-. 
righteness 4 the tedm as aged. = 


Item a: "Oid the review team allow sei time to 
swe termine comilance?” 


ho wt 90 percent answered ves* to’ titis question, and oe 


these 1% respondents Jess than half (47 percent) _ 
“iy ae rated “the laa allotment as 2 
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MERC visit? ’ ’ 
i. od a5 nercent answered yes to this yuestton, aad yl 
these 20 respondents Jess than half (85 pecan, rated | 
the ore-arientation as yood . ON 
| iter Nt “Otd questions used by MERC realty get 
at the Antéent of the ieatslatton?” 
* ° ‘ 7 Pa * : 
. we Hl percent answernd yes to this questioa, and of 
tnase J) respondents 65 parcant rated the quest tons 
. used by MERC as good. a 
Ttem 32 “dere MERC procedures fle«ible enough ta 
Lae recognize virfations tp the aovernance of 
) vocational edycation across stares?” 
‘e 30 verkent answered yes to this question, and of 
these 19 respondents 74 percent rated the. flecibrltty 
: of the MERC procedures as good. J - 
« ’ ‘ ‘ Hoos 
Item 13: “Were state staff members ‘sufficiebtiy orien: . © 
~ ted to review process and procedures tn thetr 
areas prior to review visit?’ - er # 
‘4 er ° . t . 7 y : r : \ 
20 percent ahswered yes to this question,;and <* 
re these 19 respondents less thao half (42 nercent) rated 
: tne prior orientation of the state staff as.good. P 
; ; ated J ie oS 
‘trem 14- “Did the state dtrector nrovide sufficiant : 
| —""~ Ieadershio in order to atd in adequate prens: 
' ration for compliance determination?” : 
90 percent answered ves tis question, and of 
‘these 19 respondents 74 percent*rated the Teadursh ts ; 
° of state director as 2003+ . 
i tem, 19: “Did: the co-chatrperson aive MECC team merbers j 
= ~ adequate time tc wake, a. thorducn review of; 
‘ a the areas’ for which they were responsible? 
“t my 95 percent answered yes to this question, and af 
ae “these 20 respondents 2S nercent rated rhe adequacy of 
— > time atver by the co-chairperson to team terhers a5 
7) good. . ‘ = re an . : ; att 
ee > a ap 7 ds 
4 ME 
4 a re ee 


ee 
ty 
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“Did the review tear dividesity efforts te 
adequately cover each. area being evaluated?” 


95 percent answered yes to ths questlen, ang ed 
these 20 respondents 70 percent rated the affarts (oO 
cover: eath areas as qobd. 4 


teem 10) “Were you givenan orientation erinr ta the 


item 16: “Were the secretdérial servives adequate for , 
the review actfvitias?® , 
31 percent. answered yes to this question, andof 
these 17 respondents 76 pe ent-rated the egeadecy of > 7 
the secretarial services a5 good, 
‘item U7: “were there Adee ate materials on hand fer 
you to carry oureyour Job as a MERC teat. 


mene © ay ing 
35 percent answered yes to this question, ang of * 
those ae ret rondents 70 nercent rated the ade equacy. of 


materiars foe jegm am embers as aod. |. 

tte YS: “Ape the tinw orovidiens for correc? ag non- | 
come: fance flexrbie esough to srovide far 
uMiaue st tuations , Mes. orcosigation, and - 
eattcies 3 E : 


32 Dercent aminwrred wes to this question, and of 
Teco JP respondents 60 nercent rated the time pro-- 
wisdages fo meet Comiiance an aenaye Situations as Go Me, 


whe 2 erhiveuaacee with the Hadise and recoce 
eS oe ed tains of the MERE dvaft reagent?” elie ee 


, S6 HEreeat answered yes te this question, and-¢ i. 
heya 


hese GY resconmdents €7 percent rated their agreemnnt 
with Che draft’ renert as apadg_ a “4s 
iter 20: “was there sufficient time to orepare the, 

exit reoort on-site? ct 


TG percent answered yes to this question, and. af 


th.ge if ruspondents $0. cerc ent rated thr time to 
prepere the exit report on-site as good. 


Trem 2: “was there usfficfent time to prenare the 

wee fina?! reoort 

2” percent answered yes to this ques tion, and of ‘ 

respondents most (45 percent) rated the: 

y of time to prepare the final report = : 
ate? had not received the firs) report.) oO . 


oe 


nen 22> “was the technical assistance available from 
fenronal USOE to assist states in reaching 
com tance?” . . : 


#6 percest answered yes to tats question, and o¢ 
these 18 respondents most (44-hercent} rated the techni- : 
.l assistance from regional to" rassist in reaching 
compliance as fair, 


4 , 
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Following 


', these 1? @esoondents Fi oe 


these 21 responde 


- 


< ee, —_ 


Item 23: “were the onesite MESC logistical arranqe- - 
meots suitadle?” ss le 
. a) 


100 percent nL Eariee seid maa and of = 
s 81 percent rated the suitability . 


of the logistical arrangements as good. ; we oe 
. ae 


. Res onses of Personnel in Re iona} Offices: Al) reGraicee 
of th¥s nopulati-n (N=20) are given in Table 2, Appendix C. 


are .ignificant findinas.- 


- * 
trem i: . “was the MERC review peam trained in a manner 


“TE to adequately determine compliance?” 


Hi 


‘4 gercent answered vgs to this question, and of 
retit rated the training as 


4 


4 


mood. 


Sten 2: “Was the agenda used for MERC appropriate 


45. percent answered yes to this question, ‘and of 
these 19 respondents 78 percent rated the agenda as 
good, nS ‘ 


ter 3: “Were other MERC team members courteous afd 
‘ professional in their approach to the tasks?” 


ICO percent answered yes to this question, and of 


these 20 respondents 95 percen 


rated the team's 
approach as good. °-. ‘ : 


Item 2: “Was the composition of tile MERC team members 


adequate for the job to be accomp! shed?” 
~ 95 percent answered yes to Xhis question, and of 
these 19 respondents 8° percen rated the adequacy of 
the team as good. aa bi 
. ‘| 


7 ™ ‘ a 
Item 5: "was thére an adequate number of persons ar 
7, the review heges ateewere repr ccna of 
‘regional, state, and national personnel?” 


109 percent answered yés- to this question, and of 


these 20 respondents: 80 percent rated the number and 
mre sentation of the teamas evod, wd 


Ata eagerness Secret ele a ae 


r) oP oe ——— i 
‘so> gr "Od MERC team members you dealt with know 


what they were to do to carry Qut wneir, 
role eff\ciently?”.. eo : 


90 percent a wered yes to this question, and of 
these 18 respondents most {5° per. nt) rated the 
efficfency af the team as fagr. 


! ~19- ; { : a 
| 44.0 7 


wih ceeati cate aed, ei etree in Oa 


- 


‘Item Be “Was the review team for: nright in disclo- 
e sures cf problems where ponp lance Was 


fl ‘ doubtful 2” Sing 
: i : | a 
. an: - 95 percent answered yes to this question, and of 
a _ «these 19 respondents 79 perce. rated the eee 


ness of the team as good. 


Item 8: “Did the review. team allow enough tine toe 
: deterring comp Liance?": 


- + 90 percent answered yes to this question, and of 
these 18 respondents 67 nercent rated-the time 
allotment as good. : é 


- : : s : : 
item 9: “id the review team divide its efforts tor 
adecuately cover each ared being evaluated?" - 
A 190 percent answered yes to this question, and of 
these 20 responder’s 65 nercent rated the efforts’ to" 
‘cover each area as good. r a : 


Ltem 10: “Were you given an ‘orientation orfor ta the 
“MERE visit?” os 7 . 


a ; 30 percent answered yes to this auestion, and of 
these ig respondents 78 percent rated ane pre~ 
orientation as good. 


= ee ee: 
oe tea, *Did tre questions aed by MERC really get az 
ree he tre intént of the legislation?” 
; 90 percent answered yes to: this question, and of 
thesey18 respondents: 83 percent rated me questions 
oh ustd_by MERC as mOOn 
‘Item "Were MERC procedures flexible enough to 
vocations’ education in, the governghice of 
_wocationa? education across-states/” 
. : OU percent answered yes“to this-question;-and of 


....: these 18 respondents 56 percent rated the flexibility 
"S  Of the MERC procedure as-y od. 


item 13: “Were State staff members sufficiently orien- 
me oe to..review process.and procedures in their .. 
a . areas prior to review visit?” 


tation of the state staer-as quod . 
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Item 14: “Did the state djrector provide sufficient 
- leadership iniorder to aid in adequate prep- 
aration for coup! i nce Aetermingtions ; 


90 percents areweres: yes £0 this Guest ion: and of 


s 


of the state director ag goad 


™“ these 18 respondents &3 percent rated a a al 


Item 15<°"Did. the co- chs irpers n -give MERC team members 

— “adequate time to make\a thorough review of the 

‘areas. “For which they” ere responsible?” 
a 

BS, péeeent anewered yes to 'this auestign, and ‘of. 
tnese.17 respordents ‘B2 percent rate’ the adequacy of * 
"time qiven by the co- -chairpersor to ‘team al as 
good. : y 


« 


- Item 16: “Mare -the secretarial cerugel adequate, : for 
Bk the, review activities?” ‘\ ; 


30 percent ans werd yes to this a astian, and of 

these 18 resoondents 78 percent rated te adequacy of 

‘the secretarial services as goods ; He a 

item 17: "Were there. ’ adequate mone a hand for 

you to carry. out your a0 as ‘8 i RC team 

c am member?” ; 

/ . - 3s : . a \ \ ; 7 

95, percerit answered yes to this question, and o. 

these 19 respondents 89 percent ratéd the. adequacy of 
materials ~ for team members as good. . : \% 


ak # 


‘Ltem 18: ‘Are the. time provisions ‘for ‘corregting non- 


: ae flexible enough; to nrov ide for 
-uniqué situations, taws, organizatibn, and 
policies?” got 

s ‘ ies 


ée 90 percent scaacee yes to this jquestio » and of 
sirede 18 respondents 83 percent rated- the tim pro- 
visi Ons to meet cone e in unique: eueuarion as. , 
a _ 
aes 19: “pid you agree with the’ Findings and ei 
i ‘mendations | of. the MER® draft report? : 


“4 BS sercent ansuered_yes- to-thts ~questicn, in of 
os these 17-respondents: “94 percent = “rated: per en ake 
ne the draft, EeROYe as good. ai . 
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Item 20 20: “Was: there sufficient 
oa exit Peper on-site?" 


—— ime: to phe eater 
yo 
é BY 
this aiec¥int, and of 
rated the time to. pre-. 
good.- = 


b 
y 


es 95 mercent answered yes. t 
“ pese 19 respondents 24 perce 
are ‘the exit ‘report on- “site 


. ‘tein 22: "Was there sufficiert time to ‘orepare the et 
ee : - fina} report?” : oy 


" percent. answered yes to thig question, and of | 
1%:respondents 59 percent ‘rated the’ sufficiency 
me to prepare the : “frat. report as good. 


eg mz2: “Was technica! assistance available. from 
.. - -, , Regional USOE ‘to assist’ States in ee 
ra at compliance?” oS 4 


.:, 8 eS fs BE sarcent acaba y yes to this’ question, ance 
. ~ ,  these\19 respondent’ 89: vereent rated the technical 
"7 assistance from Fegiona}. to a: sist” ‘in reaching comp] i - | 
"« ance asfgood. i . eens 
a ioe : oS “ 
i tem 23: "Were the on-site MER 
‘ ments suitable?" ~ 
« . ‘ ~“ 
100 percent. answered yes to this question, and of 
x. . these 20 respondents °0 percent rated the suitability 
Of the Teaistical arrangements , as good.,, = — 
- * ; YEE - 
Responses of Adult Education Personnel: AV" respandes of 
- this population (N=8) are shown: as Table 3, Apoendix C. , Follow 
ing are important findings. ». “ot 


seaieienh cesses 


‘s - Item, I: “aig the MERC review team trained in a’ manner e 
to adequately determirie comp] iance?" 


—— 
oom 

pene eae 

soon 


88 percent” ‘answe d. yes _to-thts-auestion, ‘and of 
theseseven respo =tess than pees percent) 
rated e-training a good. 


ial Item 2: "Was the agenda | used for MERC appropriate: “e 
a : for the tasks?" x 


ee TS" sored mannered yes to this question, and of 
“these ‘six respondents 67 pene rated the agenda as 


~ Fs . 
it. see 


a “Were other MERC team members courteous and 7 
professions in their approach to the tasks?" 


gs * 100 percent answered yes to this question, and of 
, >these eight respondents 88. percent- rated the, team's . . 
(  approacn as. eo00e - 
° “Te oe 1 7 i Mg 
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\\ . 88 percent. 


“yo: Item 4: "Was. | 


adequate for the job to be accomplished?" 


_ these seven respondents &6 percent rated the adeaery 
“of the team as good. © ; 
4 . 


5: "Was there an ‘adequate number of persons in 

tie review teams who were representative of 
: — regional , state, and national personnel?" 
: ( \ y . 


100 percent answered yes to this question, and of © 
ig these eight respondents 75 percent rated the number 
| and representation of the team _ good. 
ae ee Item 6: “Did MERC team members” you,dealt with knoe 


, hat they were to-do to ‘carry aug thielysnale 
“ - , pirictently:” 


“75 percent answered yes to this question, and “of 
these six respondents 50 percent rated the efficiency . 
of the: team. as good | énd 50. percent rated. the efficiency 
fair, . 


#8 percent answered yes. to this question, -and of 


“Was ‘the composition of the MERC team members 


“4 


~ 


; a . Item 2: “Was the review ae fort eta: in disclo- ao 
sures .of problems where compliahce was. an 
‘doubtful ?" 


, —7 


1p0. S rcenageredy yes i: this adecien and: lof 
ae __ these’ ed gt respondents 75 percent rated 


e os forth-_ 
5 _-—~¥igh*tness of the team as ‘good. ° 
oo | = 
a 1 item & 8: - "Did the review, team allow enough time to 
Pe | determine compliance?" 
te | 

| 


1C9 percent afiswered yes to ‘ehis question, and of 
18 : these eight respondents 50 percent. rated the time 
I: 

4 

{4 


allotment as good and ‘J -percent rated the time allot~ 
"ment .as fair. 


. ee 
_  ¢ tan 9: "Did the review team divide its efforts to. | 
adequately cover ‘each area being evaluated?" 
7 \ 88 percent answered yes to this aiestions. and of is 
. \ _.these five respondents 71 percent rated the. erreurs, 
i es ty cover “each area as good. 
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| oa » “se 
t 
ve 
port 
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a Item 10: “Were you given an orientstten prior : to the: 
ae TMERG VESTER 


answered yes to this giiestion, and of 
ae these seven respondents less than half 
VR 

\ 


(43 percent) 
| | rated the pre- -orientation 


: 2 os a . oe } 
. = a 135 —o oe 


> 


= ey * » : : ; : oe Lg, 
eit Item 11: "Did the guse tions used by MERC = get 
~~. at the intent of the législation?" fo 
_ 88 peveent” anewerd yes to this: aiiettiod. and of 
these séven Nesponcents 57 percent. rated the questions 
- used by MERC; "as gocd.. . —: 
Item 12: Were MERC procedures. flexible enough to 
- Pepoanize variations | in the: governance. of a 


ins. - “a education across states?” 
63 per 


i 
; answered’ yes to this question, -and o a 
‘these five respondents. 60 percent eaee the Flexibid Nyy 
of the MERC. procedure as ‘good. = oo . 
‘Item 13: "Were state staff en sufficiently“ piieds er ae 
. ted to review process” ‘and procedures. ‘in. their m5 
; areas prior to review visit?" “j 
aad 63. percent ,answered yes to this question, and of 
these five respondents less than half (40 percent) 
. rated the Mba orientation of the state staff as geod « 


Item 14: “Did the state dtrector provide “sufficient 
leadership in order to aid in. adequate pirep- * 
aration for compliance determination?”. : 


75 percent answered yes to this “question, and of — 
ie . these six respondents. 67 percent rated: the pee 
“of the state director as gcod, v 


Item 15: “Did the ‘co-chairperson give MERC ‘onde means 
adequate time to make a thorough review ofthe’ 
areas for which they were vesponsible?" Sy 

88 percent answered yes to this ‘ques tiog, and of : 
these seven respondents .100: percent rated the adequacy ‘” 
of time given by the co- ~chairberson to. team PEMBENS 

-a$ ‘good. 


a aS 


Item 16: "Were the secretarial services adequate for 
the review. activities?" 
“39 percent answered yes ‘to cis uestion, and FOO ota 
2 these seven respondents less than -hat®..{43- percent). v 
rated the adequacy. oF the secretariat “services: as good. 


Item 17: “Were there adequate . materials on hand Yor « . 
ee eee aoe ee “peer Yur 9h, -as...a MERC. team... becca 
a meinb: é au . is - 7 


a 


82 percent answered yes fo this question, dnd of 
. these séven . spondents 57 percent rated the aceauey 
"OT: materials for team: members as food. fe 3 ‘ 


vo Item 18: “Are the time provisions for correcting non-*.- 

¢ me - - compliange flexible enough to. provide- for — 

ve ‘unique situations; laws, organization, and 
. policies?" ey 2 


~ 


100 percent answered yes to this question, and of 

these eight respondents 75 percent rated the time pro- 

vision to meet comp}iance in’ unique situations ‘as good. 
ec . UE Bere, Py sh on = 


Item.19: "Did you agree with the findings and recom- 
mendations of the eee 


_ £ o-JS:percent answered yes to: this question, and of ~ 
a , - these six respondentS 83 percent rated their agree- -. 
: . -mert.with the draft report as good. : 


~ Item 203 "Was there sufficient time to prepare, the 


7 Sait report on-site?" ~ 
i ae rn a 
P88 wercent answered yes to this qvstions fe 


.» these seven respondents 71 percent rated the time to 
. prepare the exit report on-site as yood. .. 


v 


| ot final report?” > po 
oA. _ 59 percent answered yes to this question, and of .*° 
7 _ these”four respondents only 50.percent rated ‘the suf- - : 
fi ency of time to. prepare the final report as good. 


0 ue co a’ bees 

;  Ftem 22: "Was technical agsistance available from 
fee. $ . Regional: USOE t assist States in reaching” 
rp +, Compl Yance?” -\ a 


I _ Item 21: "Wag there sufficient time. to prepare the 


| _.—-~* 100 pertent answered jes. to this question, and of... 
these eight respondents only 38 percent rated “the 

technical: assistance from regional to assyst in 
reaching compl Vance 3S good. ° 


Pak su 
. —)y. Ttem 23: "Were the on-site MERC, logistical arrange: . 
-_ ‘ments suitable?". ©" * 7 
"ey *- . "X40 percent answered yes to this question. and of 


: sponses of: th 

Appendix C. | 
assessment ave given as: Table 5, Colorado; -Table'.6, Idwa; 
1 Table 7, Majne; T 


ie it ates ees ot 
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é Item: “Did you understand the purposes of MERC?" 


. 9 percent ayewered yes ‘to this question, and of 
_ these 45 respondents 84 percent rdted their under - 
standing of ‘the purposes. of MERC as good. 


Item 2; “Did ;ou understand the basis for daterminin$ 
compliance?" 


. - +) QO percent answered yes to this question, and of 
these 44 respondents 64 percent rated: their-under- ie 
standing of the basis for determining ee as 
geod, i . é 


‘ Item 3: "Did you. Understand that’ ‘the. purpose. of MERC 
; was for determining comp} fance and’ not 
qua ity?" eof, 
89 fercent answered? yes, tot this preeeiene aidiee 
at ‘these 41 respondents 88 percent rated their-under- -' 
, * ear that MERC was not to’ determine quality as 
good oe aan 
cs . a ie 2 
Item 3 4: "In your opinion, were the MERC réview teams 


a3 


ok a conpotent. to: gadequately determine comp | Lance?” 


- 93° percent: sanders yes to this ques tion; and of. 
. these 45 resrondents 84-perceht rated the competence 
pgm of the team to determine compliance (€S good. 
: Item 5: “ture MERC: team siember's “courteous and pro= — 
\ ae % f, fessional in their approach to the task?” - 

is “106 sereent answered yes to this question, and of 

‘these 46 respondents 98 perce ' rated the eoueresy and — 
prgfessionalism of the team as g008 ee 


~ Ltem 6: - "Was the size of the MERC review team adequate 
- for ‘the Job to’ be accomplished?" 7 ee 
on 7 
~ ‘ 91 percent answered yes to. this. question, and of - : 
these 42 respondents 86. percent rated the adequacy _ 7 
of the size of fhe’ team as good. ‘ 


Item 7 “Was the review: team representative of region- 
al, state, and natiqnal personnel?" 5 


\ - 


—— ee: percent answered yes to this question, and of - or 
: these 38 respondents 76 percent nated the representa - 3 
tion vot the team as iS 00h 
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Item 8." “Did MERC team ‘peabaresyan deatt with. ‘know 
; . What they were ‘to do to carry out. their’ * 
* " . role efficiently?" el 


s 


~ 


—« ©€ 3b percenit. answered yes. to. this ‘ctbtiony and. of. 
these .42 respondents 74 percent. ratee oe seettCtency 
‘of the. téam as good.. . 8 


a eH 9: "Was the ee. team forthrignt. in disclo;. 
: sures of problems where pomp tiene was 


; s ‘doubt ful?”. 
Mee . 
; ; we ie “ Qh percent answered yee to this question: and of 
ae these 42 respondents 83 ide can clei forthright- 


“-.ness of the team as “good. 


Item. 10: "Did” the fevien teani aor enough, vine to“ 
4 _ determine compliance?” ; 
.  . 93 ae answered yes to “this questioi#; and of 
vue _ these 43 respondents 74 percent rated the time atigued 
*to determine como} iance as good. - +g fae 
s : ce 
aa _ Ttem 11: "Did ithe review team divide its efférts to* 
i “4 3 cover the evaluation ‘in your area?" = Pr. 
4 183 "percent answered. yes ‘to this ‘question,’ and: ‘of: : 
~ . these 38- ‘respondents ad laps rated tlie efforts to 
. cover their area as good. a a 
‘Item 12: "Were you given. sufficient notice in advance, 
of the review?" : . 
a - 100 percent answe:.d yes to. this’ question, and of 
‘these -46 respondents 87 percent rated the eerencs, 
. notice: time as. good. 


one 13: “Did you have a pre- -review meeting: with the 


; MERC team?" 
’ 63. percent answered. yes: to we questign, aia of 
these 29 respondents , 83 perenne rated the iL 


los - meeting as “good. 
tem 14: “hh ‘the pre-review saetind. were you given : 

afl. enou,: information to prepare you “or MERC?" 

63 percent answered yes to this aiies ties and ‘of 


these 29. respondents most (52 percent) rated the 
aniount of information given at the pre- -review meeting: 


. as. fair... ~ - | 220 . 
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Item ai: “Did the questions used by MERC really get 


‘list of comp] ianck. items . prior to revi 


Item 15:" 5, the. ‘pre- pay ey méeting, did you learn’ 
- exactly why the MERC oy was being 
“undertaken?” > 


61 percent” answered yes. to this quegtion, and: of 


‘the§a 28 respondents 79.percent rated ‘their under- 


standing of: the reason for the MERC activit ty ‘as. -good. 


for review prior to a MERC visit? 


item 16 16: "Were you given a tist of ‘the scype lance Vets: 
. nm ~ - 


91 pebcent answered yes to this question, and of 
these M2 respdndents .83 percent. rated being ek a 
ten. as good. 


-Item.1?: Was tbe USOE. yidenebape. sod’ on the First 


“day. of the MERC review heipful2r ” 
A 72 “percent answered yes to this dues tion, per of. 
these 33 respondents only 52 percent rated the heip- 
fulness c* the bis tape at, ood. 


{tem 18: “Were ‘you given sufficient time during the. 
. MERC visit to discuss your area of responsi - 
bility concerning compliance?" 


‘85 percent ‘answered yés to this’quéstion, cand Of 
“these .39 respondents-79. percent rated the time - ae 


to discuss ponpitanee: as good. 


‘*. Ttem 19: “Were you ‘provided. with examples or specific 


definitions of fon '. ; 


80" percent answergé ‘Yes to this. question, and of 
these 37 respondenss768 percent rated the provis?on 
of examples as goed 


Teen 20 20:- “Were the minstionicacies oy the MERC team 
-. } appropriate for determining compliance with 
“a the requkations concerning your work area?" 


83 percent. answered yes to this question, and of 


.@ 


a 


a? 


Ja 


these: 4i_respondents 78 percent rated the appropriate- | 


ness of questions asked by the team as por 


: @t the intention of- the. sais tation?” 

: “a7 perceiit: answered yes, te this quest ion, and af 
these $C Uva 73 percent nates the quest ions 
as "good. ° : 
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. -_ beam 2: 22: “Are MERC procedures flexibje enough? to .f 

~ ey a recognize ‘variations in the governante of 
, vocational | edycation: across your state” 


£85 percent answered-yes. to this question, and of 
these 39 respondents’ 79 percent rated the flexibility. , 
of procedures as c20d. 
/ [tem 23: “Was the MERC agenda for the week ‘anpropr ate 
net ‘ for the tasks?” . : 


\ 91 percent answered yes ¢0 this guest ion’, and of 
these 42 respondents 79 percent rateg une. aPbrcariate- 
ness‘of the agenda as good. 


SS Trem. 24: "Has the Firtyrerort ber received from — 
i ae ee SOE?" 


6 


89 percent of all respondents answered no. to 

this question. a7 2 
“Ttem'25: “gid you recaige the final resort tn adequate 
time to correct compliance deficiencies?” |. 


“3 percent of se respongent> answered No to this 
Le . question. : 


ee ,item 26: "Are the exit report and fina? report 
- . 7 essentially the same?" 


67 percent of all respondents answered no to 
this question. 2 8 . . 
Item 27: “Were follow-up statements in the exit report 
e : concerning sur need tO provide i reeeon ; 
-s on compliance clear and attainable?" ‘ 


85 percent answered yes to this question, and of - 
these 39 respondents 62° fercent raked the clarity of 
follow-up |statenents good. 

oa? ‘ 
Item 28> “Did you agree with the findings and recom- — 
mendations of the MERC exit report? Bas 2 


: . : 87 percent answered yes to this question, and of 
_. these 40 respondents 75 percent rated ‘their agreement’ 
with the exit report as good. : : 


‘ ! 
Item 29: "Were vou piven a review OF tne exit report : 

— - beYore the MERC team departed?”. ~ 

_ 74° papcede ancwered yes to this question, and of ; 
~these 34 respondents 85 pefcent rateg receiving a, “« + 
Peview of ine exit report as os ' 


: : : _# hz 
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‘ [tem 30: “Are. the imac pravisions for correcting tion 
4 comp N ance Flexible enough ‘td provide for 
unique situations, laws, organiz=tion, ang: 

- § polfcies?” 
1 percent answered yes’ to this questioh,: and of: 
-28 respondents.57 percent ratéd the time Be 


Item 31 '31N°"Ts technical assistance available from.” 
L Regional USOE ta assist. states. Jn reaching 
vA compliance?” =. a 


70 percent answered yes -to tht's question, ‘and of 
these 22 respondents 66 percent rated the availabitity 
of assistance as good. - af . 


os Sone weaknesses of the process. are apparent in.these re- 
soonses. - The -training and efficiency of the team was rated as 
generally fair by Central Office and Regional Office personnel, 
and technical assistance from ere) USOE to states would seem 
to need. improvement. a . 


; : Central. Office respondents are divided on the anount one 
needed to prepare the exit report on-site. 


Al papulations rate the ore-ariéntatton meetings as needing 
itprovepent. The state personnel feel that more inforpation is 
needed at this meeting, and there is general agreement that the 
video tape presentation needs to be uograded. 


' 


; vThe ‘States had not paeeived the final report and could not 
respond to items concerning this report. State personnel also 
feel that thera needs to Ee more flexjbility os correcting none 
compliance in unique slituations. : 

_ However, it is most apparent that the number and make- -up 
of the teams was gdod. Likewise, the conduct of the teavs was 
courteous and forthright. The agenda allowed the team to cover 
each area adequately. The questians used bY MERC are good as 
they reflect the inter’ of the TSgls Ten ions: 


) State pe-sonnel understood the surodsee of MERC and felt 
they had rece ved sufficient notice in advance of- the review, 
and were sive. 300d list of compliance items prior to the 
review. 7 ; Z ; ue 
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mments - of the’ Res Responder "s 


. Part of the ‘qués tionnat re included. apen-bau ‘questions: ‘which. 


ME RE ‘précess. 


follewing questions So that areas of C 
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may be imoye mented where necessary.” 


quéstion #): 


2 a ener 


job? if 
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were degigned to” elicit optnions and: sugaestions regarding. the 


Respondents were asked to: “Blease answer the: 


RC procedures 


Following are the coments 


each popil at ‘on to these questions, 


_ what are ike two “OSE important 
be made ta allow the, wet2e procedure to 20 a better 


ry 


o' +. . . ; 
Comments of Central Office Mepcondents 
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The ttema te be eonatdered mart net inccude ttema which are. 
aleartu floor) audit gueattons, “the tyne of evidence revivded 


é 


dee not lend itself to flecal audita. 


Final reporte chould be made on a timely baste. 
‘ i} ¢ ° a 6 

Some questions need to be. revised, . 

Te prepare a degunent on the MERC process and phi losophy so 4. 
atwte agenay personnel and team membere understand and unt- 
Jormt ty extete anang the atate peraennel. (For example, 

aneé ofter aarmot be determined without a loeal educa- 
ipional agency check--compliance ts then baaed oni the eur dence 
oited oniu.} 


eomps 


' 


In-depth inatning at regi ronal project offieers meeting. 
Review areas to be looked at i tim Light of new apt. Speed up 
report and foglou-up.! : 


; 


. . : ee 
Dagal Onna! ta evaiuhetoy for program quality. \- : 
/ . 
Me down on number of pedheidai questions. : 
4 
howe that the d werement used te the same as the 7 etrienent 
prev uved with the atate on the initial viet. ; 
aie the sFidpeeihe obtained to digte Lg aehedul e team act? wrt ries 
# e > . . 
Tre final ropert from OF shouls be prepared and aubmitted to 
tht atate witetn four weeke. ’ 
Shree Blane avery peracr. in the State Department.of Vocattonal 
Edyeation necetves al ta enovled doeable of the questions : 
that be asked. Yieme ques ttoris were added that *he* 
ata rh ’s have in ome atate. 
zpmecdarr that veudd neip team to Setermine 
: ‘ a 
eae 
gtatea seudd fmes Sotrly ear 


Somers tency 


ar 
“y mere 


aT 3 
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_.. £.Advi sory Counetia 


By et ther making the whele effort a Jotnt evalua rid oe botheen 
veoational edueattow and aduit education or senarite hepa. 
> tevhas heen. (even to tre extent of thi. queationdakre) ,, 
he whole preecss was done for vocational education with 
adult: education, brought tn ac i ee t. T disapprove 
of thte whol e appr whe. 7 


fits cation of the MERC tnatrument to insure clear int 


ef. 

eS 

pretation of the rules and reguiations tn order +. eliminate 

posathle misunderstand? na on the part of the | dae ate parties. : 

-imts as to the exact intent of eaan item. ign? 3 
erever nesatble, involve state personnel in another state \e 

review Jor presortentation. . \ 


‘ 


sh Ret tan ae 
es should have been giver the opportunity for aelf- \ 


& 
fan that the faet three of mine were given. . °° 


; . \ 

should be dome oc fh a representative of this Or) 

. . = } 

* * as ~~ om Pe Pa 4 a ; Pd 
Inerecaned, involvement of AE "team! members in ali/stagee o; 
nlanning and develorrent. 2 

1 me ° s¢ eo? * ° ° 
; » individuale involved in the 


On-going MERC team with the sare ine 

ree’ ing. ‘ ; i rae 7 a ; ’ 
inetetence on having the Exéoustve Director and/or Chatrman 
respon Zz tO. ques sttone--not t. the State 


Jirector. : 


Irprove coordination between adult and voeational programe, 
earectail. in agenda and ortentation. ° F 


irproue procedure: for getting out the final report...” -\. 


The final rwpert ghouls ie taaued on exit. i 
. . ' — 


: 


Adecudte teennical assistance and follow-up should be given. 


in tnetruetional programe should be 


Expand the vompltaneo ftems te adequately cover tre law and 
réeauiation.. pe 
+ 


-_ o pe 
° 


eh, 6 os ‘ . : 
Ptveal report faex te the state tn 29 dave Prom tie 


Tan Eo 
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Comments of State Respondents: Soe, . 


. Return final report on time. 
Fe ae 


Provide more tim: to Prograns . a Pa 


Have a ‘post- meeting with MERC. 


ro 
~ 
. 


I would assign more. personnel for this kind of evaluation in 
order to -bave plent¥ of tine to present the ttems "required. 


“2 s by MERC. cs BP ow 
To tnerease the, bize of the team. cal - 
To tnel thidless in tre methodology aite vist ae to local ee 


Tha MERC rpocedute ‘should be used, at least,” every’ liao years, 
Make the MEFC more ee 1 - 
” i H 
Give participation to nore members a the ‘gtaff i 
) Give more time in oe idee performance surveys 
so as to dtscusa in full all aspects.of them. : 
: Sehedute arrangements. a ; ” ., | ‘ 


4 


Time for reports, *o be received. 


‘We .reaommendation.’. I’ felt’ that the Peview was well a 
planvied and organized. Pie 
"sé 

. A Seas wrders anding of what they are jesking for.. ; 


Provide geeaten flexit ‘~Lity. tin relation to state ee 
“Zeeal Lauai and regulations. . ar ie ® - 
. Ctari Pinaeion in, MER eaeelianee statements to reflect the 
= intent of the law, t.e., with regard to a statement in 
. the legislation which states "may" rather than "wif l"=- 

this obviously cannot he a yes on ne response. — 


~ 


Lese time. Smaltiler| staff. 


That the team be there to help the state shcw compliance.: 
> A er ee a ; & 


ft takes a Lot of time to prepare. for. .@ 
¢ More time for clarification of omy vance items ard intent. 
tena y serpreta ‘tong provi eu-mee ting. ken ee ere eer 


Ongoing procedural nettods for determining compliance, t.e. 
. more prequent COpPUET CATIONS by mato on a fiscal year basis. 


o.. 
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evaluate of am aes enate ve requirements. ee 


_ staff. 


ve a part of 


\ 


Provide for a better compliance check aé the lteveb of! 


' detual program operations (Local) in-addition to the. — 4 


fo 


Inelude interview with or questionnat res jor hover level 


MERC jorne ‘were very diffieu! to a, anil reproduction of 
forms made reading almost impossy le. Charge should be ~ 
mad" im the type for.elarity of Heading. and pep redun tier: 


Program snectalists should be given the opportunity to-mcéet 
with the. far to know more about the process and be’ ‘uble to. 
he (Process: 


of the “forms used if: the MERC was of poor 


The princin 
quact te gear contatned a few errdrs. These forms shouid be 
profes nably printed. He ee oe 


h . 
The questions asked on the forme “should relate more closely 
io the Legtslation/regulations. 


satay the video tape earlier--ct the firet meeting. : , 


sutdence that supports’ a number of aii ons should be: 
2ross-re ferenced. . 


ene fleBibilt ty and understanding ‘ More sadence notice.» 


The entt re process tn-terms of paperwork meade: to ba looked 


at. ~ The printing and reproduction ‘of papers were peor 


od 


The process takes a full week out of the work schediite of 


. the State Dyrector. - The. schedule: needs to be™ looked at | 


in terns Oo; state day-to-day activities. 
MERC team ehould have better widerstanding ea each state. 
be fore PENT ED is. made. \ << 


Satt sfied wt th the approach and 4 procedures foltoved by 
‘USOE. 


Better presentapion< on: purpose, what td# done with data ‘ 
at 0.&., people knowledgeable about apeettse area investt- 
gated on MERC team. . L 


Don't drag review out to four days when tec will suffice. 
: + 


Should look at quality of programs. + 


Pept should be-a-liztie longer eS tn rere: detat te 


She 


s “— 
: e ° 


ee 
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rather than the 


“numbers of different atare persons. 


er ieaem: 


seat 


Better meeting area, copectally iealias the working portion 
of the fy belcaae ; x accel 


Lass 


Prain team members. better so tigi know what to look for! 
Oftentimes I “had. to say, "If I were ‘determining compliance, 
I would-be "the Naan, for this." 'I had to provide pee rnee 


‘fe 


More advanced: notice. - : +t Pes 


' Better explanation in :dvance -of role and purpose of team, 


and wtett. ‘ 


Organize the MERC team according to staté structure and not 
vice versa--would save a number of teams having to 9 to 


0 


Suppl: y a final report.’ 
Hold preview meeting. 


Strengthen the team by using more state agency people coi. 
regional peoplé who are familiar with state operation. 


Be more specific 3 in kinds of aoumennaeron needed to 


. determine compli fe. 


_ Additional staff 80 ‘that more " than one person.may spend © 


equal time with each: section, rather than the spot-check., 
appneden, _ : : 


actuals more comprehensive snocititertel regarding what 


; materiale need to be supplied to eunstanti ace i a . 


Basically very post tive! 


One USOE staff member put on a hard: line sepnecan and: seemed 
somewnne oblivious-to state needs and prrege neat 


~ 


We felt the MERC procedures were still in thai formation 


stage. Improvements wee noted and were to be: adopted by 
the MERC teans later. : eo : 


Sixty- -day - follos-gh wi th final finar onoiee a. necessity--to- 
date no response has been made to our sey gees school — 


Verification of compliance. 


The opportuni ty to share wid comparé attipudes, opintons | 


_and valle eae © 


i 
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“ 
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( , ; pas MERC activities. © 


rs 


Total involvement by. regional office staff. 


an | vo, ; 
Submit a final report within a month or tuo of the eval-. 
uation visit. . 47 


Y «. 


o 


Question #2: As a MERC team member, p! ease state how you could 


ave been more adequately orenared to carry out 


Comments of Central Office Respondents : 
Designate responsiblity for the program to a single capable . 


(tkis is a must) person. 


Tt would be eaatan and more efficient to work in the program 


if operaticnal direction stemmed from sensible planrimg. ~ 
Bertes.inservicg training. 


To have been involved with a spectfie report (or tape 


recording) of all the inservice and pre-site vistt. activities. 
- (Communications) a ce 


z could have had more background data. 


Use a previous -MERC visit report as a basi , for’ orientation. 
' and clarification of tasks. Ns ~ 


Pre-visit meeting of team (in Washington) might have pre- 
pared me bet ter--mostly because of non-conststency of staff. 


Coe 


3 ‘ 
° 


“Prior to MERC the team should agree on. interpretation of 


t. 


certain key regulations.: The lack of ajreement caused 


‘| problems -am@ng team members... 
‘Needed to have more time to prepare for site visit. 


‘More time to prepare. 


More and better training. ; . _ 


* My experience wagon the first MERC so tt ie perhaps unfair 
to judge it. The pre-visit briefing was ,trtadequate., There 
‘was fo concensus among team members as to ‘the purpose of 


the. review, or even the, process. 
y ‘ ; : 
Feel I was adequately prepared. a 


Matter of worklead, but vould Like to have more time to , 


'. pevtew state's aioe! Sa file in our office. 


“ Gk tea eee ote eeneecena 84 need i a cee 


: . _. 730- 


. 


Better ingtructt. o> on what donsti tutes acceptable evidence 
of comp ili,  Thta might be hest naydled ey deve Lape nga 


team member hand so) ‘showing samples of combltance « aiden ; 


Better training prior to vteit, cepeetaily on the ‘ 
of the-etate ane organisation of its stay! ae TOREELOP Ge 
ttoris of eutdence ahd what (andrivhere to find) kinds of - 


documents or convinetng, real ‘eviderce. _ 


” 


rae | 


_ Comments at Regional Of fice ‘Respondents: a 


By Aa wins rite “MERC coordinated wt tn cither "intervention" 


aeiivities ac cready. condueted or rlamed for that atate. 
t ° oy 2 . . “7 
Prat ning : for regtoral OE people was nct prevtce eduntt? the 
evening L -ore the MERC was .to begin. Consequently, we — 
e bad . bs 
rerugited to learn tre process. . | .. ; 
: ” ‘ ” i. 
We shoul td } ave. heen eieweue exampics of evtdenec to Lock 
for, — : : 
Preparation is adequate now that sta/f tas participated. ~ 
tn at feast ane MERC euch. : <.* 
is a ( 
7 , : . : 4 
Speetfte wistruettona and examies of mintmen compliance 
ape, neededy ; a 
' * : . a 
Hapover oplantation te the process 
“aa memionrs vere aaned to taker ten had a 
portal fas ay Baa undens tims inas+ 
. : . oo ° oa ° 
ope We me.well prepared, : 
Preparation was very socc oan wes redommeriied. 
ane ; ‘ 


re. | > toy 
i “elt comrortabte in 


t 
: i i od ‘ a - 
Pe [owe asked ta byte ryNe Needed 


mone Spare Ng. 
Ina ano bhi er region ™.. 
‘ we, J on hotter 
ra ! ~ 


: ' 
2 és bid aya ag a Ce Ce See a sy 
Perrier Orreltariod ane rere EPO. Sie gutdeltines re teed ro 
: Sa ee +5 ; : ° 

DIN 2E Tare , ae OURS cians, ana Letiecr wedoyrs tandi ne ile 

wits yap me aol aati yt 
OLEALLOM OP Soe Cea €2 ay ice, : 

‘ . 
‘ 

2 eet yt the bmogeype h oop & Spey pp eeyeaes ote ey ose any 

aR IN y 4 CE SMO LRA Ue PPO SS AS at EAMAC CT AMRGBIY. . 
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More expeme REC, possthly through auditing or partietpatina 
tn MERC aetiyr ties tn another region cn at least one: oceagian 
prion te ft ret este pee tn home region. ; 


tncer my region, ae com Le ted in tote tie “iret meeting, T° 
mae not prepared stnee ‘the information: dae met avartlakie 


in navanne.. All othera fine. ° - ; 
* 0+ Well prepare d in advanee*Yue to ad experzenoe in pont tor 
Ing for, corr Itance pertodt cally tn the AE rroegran.- “ 
Le P $e ii a \ - 
eae 


Aeterna preparation vould have been @ waste of atasy time. 


a °. | ’ : 


ith represen tative of Advisory Counetis, and ques tioning 
oniv in that area, I feel the ortentat‘on was aul) ftetent 
+ te ‘i i; 
, encih ta carry out BE peaereUenye : 
Should nave been more training of all team members pricy 
to the vistt 
® SiouTd have been more interaction with the program operators, 


oe 
. . 


os 
‘ 
. 


Bu parttetpating in @ MERC review. . 
Bu spendt na some time covering the MERC questions with tear 
merbers he: ore mere with the state. : ~ 


, 


The NEPC mem>er ‘ ghnitd be involved from the initial stages 


to. follow-up : . . \ 
Ques tion #2: Was MERC helpful’ to you?’ How? ’ 


s 


Comments of State Respondents . _ am 


It heifed us to vtéw mre sneuatelsy our weaker points 3 
that wee id be able to imrreve them. 


- 


he rule 28 and regulations 


It aliowc? for a refreshing of 

: aaa ‘voogttional technical education and the knowledge | te 
of some we had overtooked.~ : 
Any sel} f-examination Za helpful. we have changed some 


cert tification statements ad applications and will change 
fa thers. The matr: areas of improvement wild be in emalt 
. technical pointe. : ym: 


ta 
Y af 


RO ' 
Ider.tifiéd areas of regulation nm wht ch we were weak cr’ | 
needed tmproverient: opper tunity to dtscuss interpretations 


of lave and regulat tone tn some depth. 


Im preparing ue for queettonable interpretations regarding 7: 
‘ _intent of law, - . 
r- , 


o_ 
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comptt tance. 


their duties and responsibilities. 


| Help other educatore | see.or understand other programés 


- The overall team effort in a ee and ; 


. Clari fuing ways and procedures aa to implementation of 


programe, and changes that can he made to meet wt th 


. 
+ 


It helped me to be aware of the laws and veailalions in 


_vocattonal education and that.our program atould bes tn 


come ciaice with the law. 


Makes the staff more familiar with rulea and regulationa. 


Provides the chance to revfew all work done and deter- 
mine the. effectiveness of the whole educational delivery 
system. 


; 
Ts a good tool to ‘ewatuaes what everybody: ts doz "9, with 


| Helped the supervisor's staff to be aware of all_ the pro- 


visions of the las, as well ae the prevat ling rules and 
regulations teaued in order to gomply with it. 


‘ - 
It made me more conscious of. the ‘importance of being tn 
compliance with the aw. . a." L 

° 3 
It makea me recogni ze and be more conscious cf the \ 


importance of gotng to the law and following it whefi 
developing proposals and implementing projecta. 


Now J am more aware of the meat tence of the Lass 


We were aware o f the depiaioisecs found out ‘and were rea 
to make the necessary planning for improvement. . We were 
éattafied ‘knowing the things we were doing in compliance 
with the regulations. 


Point out areas that can be stressed. a 


objectives and tasks. 


researching supportive dat: as well gs the recognition cf *', 
areas of ede Lice and wear-e88es tn current etate practices.' 

—- > ' 
It gave me an opportunity to review the law applicable to mj 
area of responsibtli ty and to determine: if the area was in 
compliance with the federal guidelines. which are sometimes 
vague. It was helpful to know that our bureau was in 


campliance. 


Revtew of federal legifiasion ani Maine'a activities to 
comply to the intent cf the law. ; 


ae ; | : 


me sg 37 Se te 
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. 


M™portiontly to tee an Youtatder! look In to sve ae a 

different vlew of dealing with the Afict Educa tiem fet. 

Pe faa nose te ino me cape a vac Ta Cor somp fama at oh 
fre Tait. 7 


. f 

° 
tegtela? 
—_ 


RPanovea ao erttiaal tntermal reuters. , 


af SoS expestationa regand-— 


trined a hetter underetpiding ¢ 
ing my program responatbili ties, : 

Kelped ataff yet a ketter handle on raves, regulations, 
‘and intent. . : 


» 
er a 
whe 


nent formate and Proce ‘dures hava .been developed 
fmongved the bureau's total cperattcn, 
tiem time a2. 


to 


The review aot atalf involved’ an: y forved 
look at what they vere doting. 


wader : 


Save the Burvau of vocational Educatten a better 
tained 


atanding of the rules and ? regulations as they pe 


“ecattonal Educatton Amendment _of 1968: 
1) 


tr the 


my a miner wt. Hore cogniaant of jawe and regu cttona. 


i have a rdlok bedeen Dare gu, af the finetions of ISOE, 
region, ete, an@ leva pertaining té 3 fuldins of prograrite 
provided for 
conpliance. 


review of our oum recordg: land procedures for 
Tam aated ae apot check af. our eperaras: 


tdentt fying weak arvas to be atrengthe ned. 

nae used prior to MERC will need to have 

moda te snore ton required for | 
‘\ 


oa 2 
° a . 


It wae hatpful in 
shee some of the 
a 


. 


ue self-evaluation ezerdae. a 
a ‘ 


ie oes broad compliance picture® into benter fogus 


So ug to ompt le evidence of * compliance rater 
than use explanation atone. oO 


t 
It made me review in depth the quality and quantixy cf 


my involvement. 


Indicated our “shortcomings.” ; 

a 
Preparation, sel; Satiyde, followeup, im-acrvice, the 
of an audttor. Ps 


rrenenec 


f 


wa 


. : . e . 


. - 
a 
Quated ram ta Joo gat Parte Aste Oe ome Scent tet arte re, 
Workin, to Terre, ‘ 
Sone Sami ear) he ve : 
’ ® 
a . * 
, I regiewe! aome parte af the las and Tleemae f comp (camow. : 
. J 
Therefore, we u nfgom: procedures. 
Shade ua Teak at cur portetea amd ruler ce sa. to ne mete 
aarefuliy tn ifaghe vf feder lS tas . 


feluferced the pu: i sa/preved MPed OMP agen operated with 


fourattentian to the arcaa where we verg defielent, 5 sh 
Pded a time Shame and agatetange Sor correction, 


ve, 


ve 


e 


: oo. « , ‘ . ' : 
Preperation of tie vatertals pene 20an Treacle Sea Sie 


id Me . a3 ° . . : 
te oka f atth regarda to lgatalation aid compl tanc:. 
che Woete pointed cet co weak opota and neaZfirencd cur aoe 
at reniva rea. : . 
. ss a te 


lar! fled mo compliance regulationa--Iova ta shangines 
orke-study form to comely otth To eal coment tama t. 

: t ‘ ) 
“d Mp ui O41 and review cf thd. 38 pram mt. Magy hid 
ttefeton review our operation. Tria I belteve ‘wow’ ix 
have occurred £ A she MERC team Wiad ns vtattod. @ 


+ 


prs 


ig Pree ‘dad onportuntés Sap om ch fcottye Farty to bt 


wet o. . « 
og wt Z A ee: De ce cipdegicd! Wo esge seize 
ming rf, in See opinion, Me owerg aa faot fe veep! Smee 

~oysen the far and mica and reguidtfeona. : 

‘ . ; 
oe 14 i a Pre yee we Te teryieg t .f 
we LMT LOOM fhe Sure be PMWEL ACI CE? 
the dleasuastona durine the tear viet ded me i eter . : 

as 
ft 


£ 
‘ epeyrt 
They regis cd ta: rH: Ly 


widere Bnd! sompliange pete 7 


‘ Jaos that we are movtng f.2 pee dirgottiou, The vealed ai 
% ; ; 
attention to theuehte refotinve on “aie man fe dent tote © : 
~ . 

prove our vartie + os ts aesondenee “with tie Ente: mt of thy 

legs elation ‘ : . 

Te ee a eee ce mee sae apart ees = 

Pind Le ab Paw? tPTe ce oe ade a Pal EAE as 

tng noneod epitange grier So Caer letion fe 
gifeaton: we te be tre oeerivenor arth > 

x 

: ics bi ee  aiearae, a 

wmeraile. apaaring, they were are: fond tonul and ef8iotene 
Otho ep mietomer o£ diemgeion t - datle oferetione. 

: . Fe 
. * te: 
: 
‘ a 
e 2 . 7% ¢ 
+ ; F © 
* . Cad . i “ 
om @ -355- . é 


Sa ae eee 


Ques tt 


Wiel dequatety tnifned Oorepared) pre feagron a rer neat 
ppemating tee the Sfeld, a ata degree of post tive pe Sg 
relatrena san be achrenwe! iy th eon Pe a8 tte Pe Ragen. GS 
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Estimated Cost to States a a > 


One oF the objectives of this assessment is to provide an 


estimate of the cost to the states to prepare for and partici- 


pate in a MERU review. Four of the five states visited-by the 
investigators provided manpower and cost estimates. The datz 
provided by these four states are as follows:° Be 


a S31 estimated man-hours at a cost of $5,310.00. 


*2. 643 estimated man-hours at a cost of $8,037.50. 
3. - 200 estimated man'-hours at a Wostof $2,600.00. 
4, 964 estimated man-hours at a cost. of $7,604.00.’ 


The above figures do not correlate with the size of-the - 
population of the state being interviewed in this assessment. 
One would normally. assume that a small state would spend less 
time, hence less money, on the review. However, this was not 
the cas¢. The state (#3) which: reported 200 man-hours was a 
medium sized state, and the reason given for the relatively few 


‘man-hours involved was that they- did not feel it was necessary 


to use large amounts of. time in preparation for the MERC review . 
team. Other states did usr more time to preere for the review. 
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_ CHAPTER 4 : 


a» | — 
‘ CONCLUSIONS AND RECOMMENDATIONS 


, s S soi 
Gonclusions . : . ; 


‘There is no doubt that MERU should be continued. Investi- 
gators have concluded that the MERC process provides a means- 
by whichqiSOE, Rezional, and State Offices can better assure 
complian€e with the intent of federal legislation. The MERC 
concept is held in high esteem by practically all persons inter- 


‘viewed. It. is essentially described by state people to be one 


past years. 


of the best U. S. Office activities to be undertaken over th 


Investigators tiave found that the MERC procedure S pro- 
vided a way for improvement of compliance at the staf® level 


and at the same time has brought Regional and Central Office 


people into the fie'd to séé first=hand the variety of problems - 
faced by individual states which are trying to comply with 
national regulations. a ae. . os 


The process in general is adequate to attain the objectives 
of MERC. The following recommendations are give.. for consider; 
ation in order to improve the MERC process. These recommendations 
are not given with any suggested priority. 
Recommendations yw? 3 a 

1. A procedures manual must be developed to provide review 


_ team members with a guide. for use in each state so that-con- 


sistency can be attained for determining compliance and carrying — 
through the daily review activities and exit.procedures. .. . 


2. There snould be a thorough in-service training program 
for Central Office and Regional Office. personnel on the objec-— 


tives of MERC -and the procedures for conducting reviews. — * 


3. There should. be a. director of the MERC review team - 


“instead .of the present co-director structure. The director : 


should ‘be selected from the Central Office staff. 


a It is imperative that Central Office and Regional per- - 
sonnel! who meet with states: for pre-review activities be the 
same persons that meet with states during the regular review. 


S.° Every opportunity should be provided for a state 
director to be.an observer of an entire state review before 
his state, review is held. _ _ 


6. A means should tie provided to insure tiat Adult. Educa- 
tion, Title IX; and CETA Pesrecentatiod will be included as an 
« integral part of each state review. 


7. A thorough orie tation session must ie conducted for 
MERC team members before’ ney go into the field. 


- 8 A permanent central office team should be developed to 
tarry out all future reviews. . 


9, It is essential that the’ ‘State director.be given a 
written statement on non-compliance. items each day of the 
review. ‘ 


yo. eee MERC participants should have an o portunity 
‘to review the exit report findinas concerning items of "rmon- 
compliance” and tems “to be determined" before the MERC team 


leaves the site. 


11. Each state should receive a written final report within 
39 days of the review team exit, he teteebestey et ok cae 


‘12. A new video tape, should b developed which will be 
- appropriate for the orientation of state personnel in ail pro- 
grams. This orientation tape should include an overview of thé — 
purposes of MERC and utilize a variety of charts, jgraphs, and 
. visuals which will better serve the entire viewing population. 


_ 13. Effort shopld be made by USOE staff to systematize the 
in-house. MERC repor® activities and to provide adequate logis- 
tical support to produce the completed document within the 
recommended 30 days. ; 


14. At least | once. a year, Central Cffice permanent review 
tedm members should identify compliance questions which should 
be deleted or rewritten for clarity. 
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APPENDIX A 


Interview Form for . 
Central. Office and Regional Office Personnel 
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lt “ _ MERC TEAM QUESTIONNAIRE FOR - 
EVALUATICN.OF “ERC PROJECT . “ 
Bate 
Identification - lease check one: 
__Recizna? Office "| ____Deputy Commissioner's Staff 
DEE Cther (please specify) 
Tn tA . ve = _ ¢ 
“TTTAdul Education ee & he ; 
Oi- sections - Answer eich of the following questidns “yes” or “no. if yore answer 
is “yes,” rate your respons® as “good,” “fair,” or “jot 
RRR Sh " a - - " err reese ene Wt n Roba itl 
Ko Question . 4 0d ‘air | Poor 


1, Was-the MERC review team: trained in a ‘ere to” “8 ; 
- , adequately determine. compHlanes? . 
Same: Was the agenda used for HERC appropriate for the tasks? =e 


. Were’ other. MERC team members courteous and professional 
. ty their puree to the task?’ . 
4, Was the cvenatisiah of the. MERC team members adequate 


for the job-to be accomplished? 


5. Was there an sdequate number of persons {n the review 
teams who were representative of regional, ddan) and 
national personnel? : 


6. Did KERC team members you dealt with know what they 
were to do to carry out their role efficiently? 


. 
. 


2. Was the review team forthright fn distlosures of 
problems where compliance was Goubtful? _ 


“8, Did the review team allow enouch time. to secermne 
como] fance? ee 


» 


9, oid the review tea divide {ts. efforts to adequately 
cover each area being evaluated? . \. 
& 


10. Were you aivet an orfentation prior to the MERC visit? 


Ll. Did the aimetloné ysed by/FERC really get. at the 
- intent of the legfslation 


od 


eer inter aeemmaneaie bean Paaqeeerne. 
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: . view oroiess and erocedures is ee ee aris 
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4 | i 
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7 H 
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neh tecnrscal assistance 202: ilatte, from Regional USOE 
to assist states oe reaching corgliance?. 
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___,Durtng the fall of 1976 the Bureau of Occupational and Adult. ... 
Education, U. S. Office of Education, contracted with Sierra Plan- — 
ning and Research Associates to conduct an evaluation of the MERC 
process. ‘The objectives of this assessment were: (1) to deter- _ 
Ze mine the feasibility of continuing the program, (2) to determine. 

i “ways to improve the process, (3) Ydentify cost to states for the 

. ©. review, (4) identify benefits to the states, and (5) identify: 

“ .benefits to USOE. ° . 3 er 


_. Utilizing a structured interview approach, SPRA contacted ., . 
- personne? in five of the. 13 states. which had been revi . by, 
~' -MERC. SPRA interviewed personnel in six Regio aTPOr Nees at 
"” and various /persongel from the Central USOE. Additionally, a 
SPRA observed the C review in idaho. ve a 


a | ‘ 


5 pees ps . ee ee 
A total of 87 persons. were: intereviewed by*SP, »)and data, 2 om i 
fram thdse various sources: yiel 14 recommendation which are \ 


catégerized as fel lows : - 8 ny, 
lL The MERC process tho 1d be, better organized. oe 
2. : Communications should ‘be improved among all 
*- personnel. : 
3. The MERC process should be reviewed periodically 
to remove or alter unsuitable items, © 0 
a a "_. 4, The process should be systematized through the 
2 . eG development and usage of a procedures manual. 
The: general conclusion of the fnvestigators is that the . 
. MERC process should be continued on a regular basis as‘prac- «<-~ 
tically all the people interviewed held the concept of MERC 
: _in esteem. ; we ; . Pg . } 
. . Y . * 
» | ° , an 
. eo 4 
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